Skip to content

Conversation

@jimczi
Copy link
Contributor

@jimczi jimczi commented Aug 20, 2025

System indices with non-dynamic mappings normally reject put mapping requests that attempt to modify the mapping of a managed index.

However, for fields like dense_vector and semantic_text, some mapping options depend on the shape of the ingested data. The first ingestion often requires a dynamic update to set these defaults.

To support this for system indices, this change marks the relevant put mapping requests as originating from bulk ingestion, which bypasses the system index mapping check. This is safe because:

  • such updates are already validated against invalid dynamic updates, and
  • an error would be raised earlier if a new field is added when dynamic is disabled.

Fixes #133171

@jimczi jimczi added >enhancement :Data Management/Indices APIs APIs to create and manage indices and templates v9.2.0 labels Aug 20, 2025
@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Aug 20, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@elasticsearchmachine
Copy link
Collaborator

Hi @jimczi, I've created a changelog YAML for you.

@jimczi jimczi requested a review from pugnascotia August 20, 2025 20:50
@dakrone dakrone added :Core/Infra/Core Core issues without another label and removed :Data Management/Indices APIs APIs to create and manage indices and templates labels Aug 20, 2025
@elasticsearchmachine elasticsearchmachine added Team:Core/Infra Meta label for core/infra team and removed Team:Data Management Meta label for data/management team labels Aug 20, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

System indices with non-dynamic mappings normally reject `put mapping` requests that attempt to modify the mapping of a managed index.

However, for fields like `dense_vector` and `semantic_text`, some mapping options depend on the shape of the ingested data. The first ingestion often requires a dynamic update to set these defaults.

To support this for system indices, this change marks the relevant `put mapping` requests as originating from bulk ingestion, which bypasses the system index mapping check. This is safe because:

* such updates are already validated against invalid dynamic updates, and
* an error would be raised earlier if a new field is added when `dynamic` is disabled.

Fixes elastic#133171

naming
@jimczi jimczi force-pushed the system_index_dynamic_default branch from fa20dc3 to 9705810 Compare August 20, 2025 21:01
@alexey-ivanov-es
Copy link
Contributor

A couple of questions to clarify:

  1. Does this only affect put mapping requests that are generated from dynamic mapping update during indexing?
  2. Why "bulk"? Is it always from bulk requests? And I couldn't find where this is actually checked in the code - can you point me to where the bulk origin is checked?

@jimczi
Copy link
Contributor Author

jimczi commented Aug 22, 2025

Does this only affect put mapping requests that are generated from dynamic mapping update during indexing?

It is, see below.

Why "bulk"? Is it always from bulk requests? And I couldn't find where this is actually checked in the code - can you point me to where the bulk origin is checked?

Yes, see

mappingUpdatedAction.updateMappingOnMaster(shardId.getIndex(), update, mappingListener);

It is called as part of the bulk action if a dynamic update is needed.
For the check, see
The idea is that put mapping requests are not checked for system index violations if they are generated by the system indirectly.

Copy link
Contributor

@alexey-ivanov-es alexey-ivanov-es left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, LGTM

@jimczi jimczi merged commit a560919 into elastic:main Aug 26, 2025
33 checks passed
@jimczi jimczi deleted the system_index_dynamic_default branch August 26, 2025 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Core/Infra/Core Core issues without another label >enhancement Team:Core/Infra Meta label for core/infra team v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Semantic text mapper restriction specific to system indices

4 participants